feat(fastsync): /fastsync/anchors feed + concurrent (SQLite/WAL) bundle store (#115)#124
Merged
Merged
Conversation
…le store (#115) Completes Option B of the fast-sync overlay (epic #111): the verifiable anchor-bundle HTTP feed, plus the concurrency fix that unblocks it. - internal/fastsync/store.go: convert the bundle store from bbolt (single-process exclusive lock) to SQLite/WAL, so the `start` writer and the `serve` reader can open it concurrently — the same property anchors.db relies on. Same Put/Get/Range API, so #113's indexer wiring and tests are unchanged. Still fail-closed / no reorg rollback (a stale bundle fails the client's merkle check; re-index overwrites). - internal/fastsync/service.go: Service — the transport-agnostic serving core (shared with Option C #122). AnchorBundles(from,to) reads the store, clamped to a max height span + max bundle count. - internal/api: GET /fastsync/anchors?from=&to= → {"anchors":[{height,txIndex,rawTx hex,branch hex[]}]}, CORS-open. The client reconstructs each bundle and verifies its merkle proof against its OWN PoW headers — the source can only omit, never forge. - cmd/ion-node: --serve-fast-sync now both (a) makes `start` capture bundles (WithBundleWriter over the SQLite store) and (b) makes `serve` expose /fastsync/cas + /fastsync/anchors. node.close() closes the bundle store. To populate bundles in production, run `start --serve-fast-sync` (captures going forward; a re-index backfills history). Tests: Service range/order/bounds; two concurrent Store handles (the start/serve processes) read+write the same file (a bbolt store would fail the 2nd Open); and the /fastsync/anchors endpoint serialized a bundle that a client reconstructs and VERIFIES after the JSON round-trip (+ CORS, bad-range 400, route-absent 404). go test -race ./... green. Co-authored-by: Liran Cohen <liranlasvegas@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes Option B of the fast-sync overlay (epic #111): the verifiable anchor-bundle HTTP feed, plus the concurrency fix that unblocks it.
The concurrency fix
serve(reader) andstart(writer) are separate processes. #113's bundle store was bbolt — a single-process exclusive lock — so they couldn't both open it. This converts the bundle store to SQLite/WAL (the same propertyanchors.dbalready relies on for concurrent start+serve), keeping the exactPut/Get/RangeAPI so #113's indexer wiring and tests are unchanged. Still fail-closed / no reorg rollback (a stale bundle fails the client's merkle check; re-index overwrites).The feed
internal/fastsync/service.go—Service, the transport-agnostic serving core (shared with Option C fast-sync C: native P2P messages via a vendored btcd wire (getanchors/anchors, getcas/cas) #122).AnchorBundles(from,to)reads the store, clamped to a max height span + max bundle count.GET /fastsync/anchors?from=&to=→{"anchors":[{height,txIndex,rawTx hex,branch hex[]}]}, CORS-open. The client reconstructs each bundle and verifies its merkle proof against its own PoW headers — the source can only omit, never forge.--serve-fast-syncnow both (a) makesstartcapture bundles and (b) makesserveexpose/fastsync/cas+/fastsync/anchors.To populate bundles in production: run
start --serve-fast-sync(captures going forward; a re-index backfills history).Tests
Servicerange/order/bounds.Storehandles (modeling the start writer + serve reader) read+write the same file — a bbolt store would fail the 2ndOpen./fastsync/anchorsendpoint serialized a bundle that a client reconstructs and VERIFIES after the JSON round-trip (+ CORS, bad-range 400, route-absent 404).go test -race ./...green (28 packages).Post-Deploy Monitoring & Validation
start --serve-fast-sync,bundles.dbgrows as anchors are indexed;serve --serve-fast-syncanswersGET /fastsync/anchors?from=&to=with bundles a client can verify. The two processes sharebundles.dbvia WAL./fastsync/anchorsdespite indexed anchors →startwasn't run with--serve-fast-sync(no capture) or history needs a re-index backfill.Part of #111. Closes #115.
🤖 Generated with Claude Code